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ABSTRACT 


In an evolution from academic journal prestige ranking 
procedures used in economics, marketing, sociology and 
previous efforts in information systems (IS), this study 
developed prestige weights for IS journals based on a 
probability sample of 400 MIS faculty in the United States and 
Canada. The weights are derived from subjective rankings by 
survey questionnaire. In addition to developing a master 
schedule of journal prestige weights, differences in prestige 
weighting are examined according to location of respondents' 
employing school, respondents' academic specialties within IS, 
types of degrees held by the respondents, and years in which 
respondents earned terminal degree at a graduate school with 
a national ranking. Characteristics of respondents are 
compared to nonrespondents to test the hypothesis the 
respondents are representative of the sample frame. 
Implications of the procedure developed here are discussed in 
terms of its application to evaluation of research 
productivity in DOD laboratories. 
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I. INTRODUCTION 


A. BACKGROUND 

Although they are not quantified, subjective notions of 
the differential prestige of academic journals are a major 
factor in decisions about hiring, promotion and tenure for 
information systems (MIS) faculty. Rather than evaluate a 
faculty member by the number of published articles, decision 
makers are drawn to judge a publication record in terms of 
the prestige of the particular journals in which articles 
appear. This weighting process is subjective and 
impressionistic. Borrowing from the work of reference 
disciplines, especially sociology (Glenn, 1971), this study 
develops a schedule of prestige weights for IS journals. 

The weights are based on a survey of 400 MIS faculty listed 
in the 1992 Directory of Management Information System 
Faculty. 

The schedule of prestige weights is provided for 24 
journals from MIS and related fields. We examined the 
differences in the weighted ranking of IS journals according 
to the respondent's prestige of university, region, academic 
specialty, degree type, and degree date. 

Prestige of IS journals is based on perceptions of 
academicians in the MIS field. Many methodologies (e.g. 
citation analysis, ranking, weighting) have been used by 
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sociologists (Glenn, 1971, and Shamblin, 1970), economists 
(Laband, 1985, and Medoff, 1989), and MIS professionals 
(Shim et al., 1987, and Ramesh and Stohr, 1987) to obtain 
journal rankings. 

B. OBJECTIVE AMD SCOPE 

The objective of this thesis is an extensive examination 
of the methodologies used in quantifying the impressionistic 
value of prestige. A survey of MIS academicians will be 
used to determine a weighted ranking of IS journals. Five 
independent variables will be used in conjunction with the 
survey data to determine if they have any effect on the 
weighted ranking or response rate. 

With DoD's recent interest in re-engineering business 
processes, this study provides a tool for measuring 
performance evaluations. DoD has the continual problem of 
evaluating a manager's performance, in many respects a 
quantification of abstract qualities. The methodology used 
in our study has significant implications in terms of its 
application to evaluations of research productivity in DoD 
laboratories and schools. 
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II. LITERATURE REVIEW 


A. BRIEF BACKGROUND 

The field of Management Information Systems (MIS) has 
grown in proportion to the technological advances in 
computer hardware and software. Concurrently, universities 
have opened MIS departments and are expanding their academic 
staff for the purpose of teaching and doing research. As in 
most fields, article publication is an important part of the 
hiring, promotion and tenure processes. A 1980 survey of 
deans associated with the American Association of Collegiate 
Schools of Business (AACSB) indicated that publication was a 
requirement for promotion and publication in a journal of 
merit has a definite impact on hiring decisions. This is 
supported by research done by Mabry and Sharplin (1985). 

They found that the quality or prestige of the publication 
had an impact when promotions were considered. 


B. REASON FOR RANKING JOURNALS 

Determining which journals are prestigious is important 
in the MIS discipline. A list of IS journals, ranked 
according to mean weight will aid faculty who are in the 
position to make hiring, promotion, and tenure decisions. 
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C. BASIS FOR THE RANKING 

Doke and Luke (1987) equated prestige with quality. 
Prestige, being an abstract property, is defined in the 
Webster's Unabridged Dictionary as "standing or estimation 
in the eyes of people; weight or credit in general opinion; 
subject attribute, often used interchangeably with quality." 
In studies by Lewis (1968) and Knudsen and Vaughan (1969), 
prestige was considered different from quality, both in the 
way it is measured and its conceptual rather than concrete 
existence. 

Shamblin (1970) contends that objective tests will give 
a measure of quality while subjective tests give a measure 
of prestige. He believed that some faculty members may 
think a journal has qualitative value, but is not 
necessarily prestigious. Lewis (1968) found a disparity 
between objective and subjective rankings. He criticized 
prestige rankings by saying they are meaningless outside the 
immediate community. They do not relate to dependent 
variables of the material world as closely as objective 
criteria. On the other hand, Knudsen and Vaughan (1969) 
suggest some relationships between the objective and 
subjective results of surveys. 

Doke and Luke (1987) did a survey on the quality of IS 
journals among the faculty of business schools. The 
instructions were to rank the top ten journals in order of 
decreasing importance. An importance/prestige index was 
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created using a formula for the number of times a journal 
was given the same rank by respondents. This survey used a 
formula value for prestige. However, this is not an 
effective measure of prestige, since prestige is abstract 
and quality does not necessarily equate to prestige. 

Regardless of how journal prestige is measured, it 
affects decisions about hiring, promotion, and tenure. 
According to Gordon and Purvis (1991), the common goal of 
determining a ranking of journals in any of the disciplines 
was to provide university review committees with one fairly 
objective measure of "research excellence" for use in 
promotion and tenure decisions. 

1. Sociology 

Studies in the field of sociology provided varying 
methodologies for gathering information to ascertain quality 
or a ranking for sociological journals. The journals were 
ranked by a variety of methods. Citation analysis (the 
number of times a journal is cited) was the most common 
method of determining the journals' quality. This method 
does not prove to be the best for the following reasons: 
(l)the size of the journal will determine the number of 
potentially citable items; (2)a longer established journal 
will have more citations than recently established journals; 
(3)journal circulation, dissemination of reprints, 
availability in library collections and coverage by 
secondary indexing and abstracting services will influence 





citation patterns; (4)a few articles that are highly c ted 
may distort the citation patterns for a journal; (5)journal 
article length and number of references may influence 
citation patterns; (6)journal prestige differentiation will 
influence citation patterns ar.d (7) the reputation of authors 
and the controversiality of subject matter published in the 
journal will influence citations patterns and a growth in 
the literature (changes in the number of journals or 
articles published) will influence citation patterns. 

Glenn (1971) evaluated the journals for prestige 
using a list of journals and arbitrarily assigned a 
benchmark weight to one of the journals. A benchmark weight 
of ten was used to keep a relative frame of reference on the 
numbers used in the weighting. For example, a respondent in 
the sample, who liked a journal as much as another 
respondent could use a wider range of numbers (1 to 100 
instead of 1 to 10) to express his or her opinion even 
though the journal was equally liked by both respondents. 

By assigning a benchmark weight, the two respondents will be 
forced to weight all the journals in accordance with the 
pre-assigned weighted journal. 

Glenn (1971) sent the survey to faculty members 
randomly drawn from the 1969 ASA Guide to Graduate 
Departments of Sociology. This sample was representative of 
senior faculty who would be involved in the hiring, 
promotion, and tenure decision process, as well as the 
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junior faculty who would be using the journals for research. 
The survey yielded a weighted prestige ranking of the 
sociology journals. Glenn (1971) believed that any sampling 
biases that might have occurred in the faculty due to 
geographical region, prestige of university, differing 
degree types, and age groups was addressed by using a random 
sample. 

2. Economics 

The literature on journal's prestige in economics 
falls into two categories: (a)those dealing with the 
publishing performance of economists and their departments 
and (b)those concerned with the prestige value of journals 
as a measure of research performance. Exploring the methods 
of obtaining the prestige value was useful in determining 
which methods were to be used in ranking journals within a 
specific discipline. (Medoff, 1989; Gibbons and Fish, 1991) 

Citation frequency was commonly used to determine 
rankings of finance and economic journals. These citation 
frequencies were multiplied by factors based on previous 
studies using a subjective rank ordering of scholarly 
journals in the discipline based on their perceived quality 
by readers (Ellis and Durden, 1991). As an example, Mabry 
and Sharplin (1985) based their study on the relative 
importance of journals used in finance research on a 
previous survey by Coe and Weinstock (1983). Coe and 
Weinstock (1983) surveyed department chairs to seek a 
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subjective ranking of finance journals. However, there were 
some flaws in the survey; it was short and had an incomplete 
list of journals. It was left to the respondents to fill in 
additional journals that they deemed as important. 

According to Mabry and Sharplin (1985), "either no initial 
list at all or an extensive one should be provided to avoid 
bias toward listed journals." Furthermore, "while the 
impressions of journal quality by department chairs are 
useful in certain contexts, academic administrators 
represent a small, specialized group and may be poor 
surrogates for all academic researchers." 

Even though survey methodologies may not always be 
the most appropriate measure of journal quality due to the 
bias imposed by the design of the questionnaire or the 
groups selected to judge the journals, they still provide 
the subjective perceptions of significant faculty with 
respect to the relative quality of academic journals. 
"However, the most objective measure for addressing journal 
quality as most scholars perceive it is the citation 
frequency of published studies." (Mabry and Sharplin, 1985) 
Citation frequency is an objective technique for evaluating 
the impact of scholarly research. It has been used more 
frequently than subjective survey techniques (Mabry and 
Sharplin, 1985). 

A study by Ellis and Durden (1991) identifies 
citation frequency as one of the two most important factors 
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that influence an economist's perceptions of journal 
quality. A journal's quality can reflect the impact its 
articles may have disseminating knowledge within the 
discipline (impact measured by citation frequency; Liebowitz 
and Palmer, 1984). However, a journal can still maintain a 
certain level of prestige among colleagues in the profession 
well after its citation frequency declines. 

Economists have mixed feelings regarding the 
measurement of research quality. Ellis and Durden (1991) 
and Liebowitz and Palmer (1984) believe strongly in citation 
frequency. Others, such as Medoff (1989), criticize 
citation counts as a measure of research quality, especially 
of individual economists. First, counting pages or articles 
in journals ignores the contribution that books may make to 
economics. Second, some articles may be published in well- 
known journals but may not be used for research or worse 
yet, even read. "Publication in a major journal does not 
necessarily measure the impact of the work, its creative 
insights, or its seminal contribution," Medoff (1989) 
explains. Therefore, articles in well-known journals are an 
imperfect measure of quality. And third, there is always 
the possibility of bias in quantitative publication ranking. 
Laband (1985) found that on average over two extra pages are 
falsely credited to authors affiliated with the same school 
as the editor of a journal. 
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Hawkins et al. (1973) explored the notion of 
implicit prestige ranking attached to a journal. Their 
initial questionnaire was sent to 160 academic economists 
which were stratified into four groups of 40 each. The 
groups were: (1)deans and department chairs at large 
research-oriented universities; (2)faculty members at such 
universities; (3)individuals with major administrative 
responsibilities at smaller teaching-oriented colleges; and 
(4)faculty members at such colleges. The sample was also 
stratified by age group and academic department. 

Out of the initial 160 questionnaires sent, 111 were 
returned. The statistical information was sent back to the 
respondents (with a reminder of how they ranked the journals 
the first time) in a second Delphi wave. Ninety-two of 
those were returned. In the second survey, eighty percent 
of respondents changed at least one point score. 

The study also reported a familiarity index because 
the percentage of respondents who reacted to each particular 
journal were to give a point score only if they were 
familiar with a journal. An alternative prestige ranking 
based on the mean point score times the familiarity index 
was shown, this gave an equal weight to perceived quality 
and presumed readership in establishing the prestige of a 
given j ournal. 

The researchers also sought to reveal the prejudice 
that economists have toward general, well-known journals. 
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They included two fictitious journal names to test the 
reliability of their analysis. The fictitious Journal of 
Economic and Statistical Theory (J.E.S.T), ranked in the top 
third of all journals surveyed (twenty-fourth out of 87). 

The other fictitious journal, entitled Regional Studies and 
Economic Change (R.S.E.C.)--obviously a more applied and 
specialized journal, but equally nonexistent--ranked in the 
bottom third. (Hawkins et al., 1973) This demonstrates an 
ego attribute in journal ranking. This is that people do 
not like to admit their own ignorance or are careless when 
reviewing the titles, merely marking off a journal name if 
it sounds familiar or scholarly. 

Gordon and Purvis (1991) combined research from a 
variety of disciplines to measure research performance in 
industrial relations. The disciplines include economics, 
industrial psychology, organizational behavior, and 
sociology. The study discussed three ways in which journal 
quality may be measured. First, by surveying a 
representative sample of members in the discipline to 
construct a hierarchy of journals. Second, by using 
citation indicators to rank journals. And third, consulting 
r'' , ceptance rates of articles into journals. Cabell (1988) 
reported the acceptance rates for hundreds of journals in 
industrial relations. One should keep in mind that 
"although acceptance rates are a fairly stable property of 
journals, they may not correlate significantly with prestige 
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ratings." (Hargens, 1988) After reviewing the three ways to 
measure quality, we concluded that surveying a 
representative sample is the best course for constructing a 
hierarchy of journals. 

3. Management Information Systems (MIS) 

The number of IS journals used in the discipline is 
growing and the number of articles referring to MIS matters 
in non-IS specific journals is also increasing (Hamilton and 
Ives, 1982, Ramesh and Stohr, 1987). Business journals 
include MIS topics even though they are not solely devoted 
to MIS. Shim et al. (1987) and Davis and Fry (1986) did 
studies which ranked IS journals, however, their studies 
used only five and twelve journals, respectively. 

Other methods of ranking have been attempted. Koong 
and Weistroffer (1989) ranked the top 15 MIS journals from a 
list of 70 journals. They asked colleagues to choose the 
three best journals for reading and the three best journals 
for publishing. This was not necessarily an indication of 
quality. If a certain journal has a higher article 
acceptance rate than another, then more colleagues will 
respond favorably to that particular journal for publishing, 
it does not mean they have a high opinion of the journal for 
reading purposes. 

Nord and Nord (1991) used the results of three other 
studies to pick the top IS journals. IS journals common in 
the top 15 of all three studies were picked as the top five 
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IS journals. This procedure gave the MIS field only five 
top journals. But a larger number of journals are currently 
being used and need to be recognized in the MIS field. As 
mentioned previously, Koong and Weistroffer (1989) listed 
over 70 IS-related journals. Therefore, limiting the number 
of top journals to five does not give the selection or 
variation needed for studying the growing MIS field. 

Although the 70 journals were not dedicated solely to MIS 
issues, they still provided information to MIS colleagues 
as well as those affiliated with related disciplines. 

Hamilton and Ives (1983) did a six measure journal 
stratification resulting in four strata of journals using a 
composite indicator of journal importance. The journals 
were split into four disciplines; MIS, Computer Science, 
Management Science, and Management. Since the MIS 
discipline is interwoven in other disciplines, the 
distinction can not be made between the material within the 
journals of each discipline. Because the articles are 
interchangeable, not belonging exclusively to one 
discipline, it is not imperative to have separate categories 
for ranking purposes. 

Nonetheless, in the Hamilton and ~ves (1983) study, 
the surveys went to a senior group of "knowledgeable and 
recognized experts" in the academic MIS community. It was 
believed they could provide a more informed judgement about 
journal contributions than junior faculty. However, many 
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less-known MIS faculty are involved in promotion and tenure 
decisions. Also, junior faculty are active in publishing 
(often more than senior faculty) and are more anxious about 
publishing in prestigious journals. Senior faculty either 
have stopped publishing or can publish with ease in any 
journal because they are well-known faculty statesmen. They 
are less sensitive to the current prestige of journals. 

They would respect the opinion of their honored colleagues, 
but probably do not know what those opinions are. They have 
their own opinions about journal prestige which would be an 
influential factor in their decision processes. 

Ramesh and Stohr (1987) used two separate surveys to 
study journal prestige. The first was sent to journal 
editors and conference chairs to understand the refereeing 
and acceptance process. The second was sent to 200 
researchers in computer science (100 obtained from a list of 
department chairs at major United States and Canadian 
universities, the other 100 eminent in computer science 
research areas) and 200 researchers in MIS (200 obtained 
from the 1987 McGraw-Hill Directory of Information Systems 
Faculty) . Their aim was to measure the perceived research 
quality of 33 journals and 14 conference proceedings. The 
study shows, once again, how such perceptions influence 
hiring, promotion and tenure decisions. 

Although a number of publications were related to 
the MIS field, some were not necessarily core IS journals. 
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Ramesh and Stohr (1987) also noted the inherent problems of 
opinion surveys; they are void of objective facts and suffer 
biases, while measuring subjective perceptions. Computer 
science respondents returned forty-two percent of the 
surveys and MIS respondents returned sixty percent of the 
surveys. Most of the respondents were full professors. A 
familiarity question ranging from "don't know" to "know 
well" was included in the survey. This was an attitudinal 
question about behavior. A research quality question 
ranging from "poor" to "excellent" was also part of the 
survey. Those respondents who claimed no knowledge of a 
publication but gave it a quality rating were excluded from 
the analysis. In a comparison with Hamilton and Ives (1983) 
and Shim et al. (1986) studies of the top ten MIS journals, 
Ramesh and Stohr's (1987) findings were in agreement with 
eight journals appearing in both studies. Even though 
Ramesh and Stohr (1987) provided information about the 
perceived quality of MIS journals, they noted that "the 
relative rankings of the publications ... indicate an 
academic field that is still in its formative stages." 


D. CONCLUSION 

In every discipline there are two distinct career paths 
to follow. One pursues the discipline as a professional or 
as an academician. Both careers need the most current 
information available in the field, but may use publications 
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in a different manner (Babchuk and Bates, 1962). This study 
showed that professionals referred less often to journals 
and publications than do academicians. Therefore, IS 
academicians are best suited to weight IS journals. 

In addition, a study modeled after Glenn's (1971) 
sociological study on perceived journal prestige by 
weighting has not been done in the MIS field. Such a study 
would provide reference by which the vita of candidates for 
hiring, promotion and tenure could be judged. 
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Ill.METHODOLOGY 


A methodology influenced by the work of Glenn, (1971) 
was used to obtain a ranking of IS journals. It identifies 
prestigious IS journals through a survey of MIS 
academicians. The findings in this study are based solely 
on the respondents perceptions of IS journals. 

A. TEST SURVEY AND QUESTIONNAIRE SELECTION 

The survey methodology was narrowed down to two 
different forms: (a)FORM 1 included a cover letter with 
instructions to rank the top ten journals from the list of 
IS core journals. The journals were to be ranked (one 
through ten) in accordance with each colleague's judgement 
and (b)Form 2 included a cover letter with instructions to 
assign weights to the list of IS core journals. 

A test survey of each method was conducted among members 
of the MIS staff at the Naval Postgraduate School (NPS). A 
list of 20 IS core journals was obtained by using a 
consensus of journal names from Hamilton and Ives (1983), 
Davis (1980) and Vogel and Wetherbe (1984). The twelve 
member staff first received FORM 1 (Appendix A) and three 
weeks later received FORM 2 (Appendix B). 

On FORM 2, Management Information Systems Quarterly 
(MISQ) was selected as a standard of reference, and was 
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given a weight of 10. Other journals were to be weighted 
accordingly. 

Comments associated with FORM 1 (the ranking) were: One 
staff member noted a need for more categories of journals 
(e.g. Research, Theoretical, Application, Computer Science, 
and Management). Another found the familiarity issue a 
problem. Respondents were uncertain how to respond if a 
journal was unfamiliar; give it a low score or none at all. 
There was also a sense of curiosity associated with the mix 
of scholarly and trade publications. 

FORM 2 (the weighting) comments were: One staff member 
said that it would have been easier if we stated the point 
of view of which the grading was based (e.g. readership, 
authors). Weighting the journals using MISQ as a reference 
seem to confound some staff members. They felt 
uncomfortable assigning weights as two times or one half as 
important as MISQ which was given a ten. One concern which 
was universal to both surveys was that the list of IS core 
journals was not inclusive. The following journals were 
noted as missing from the list: Decision Support Systems,- 
ORSA Journal on Computing; ACM Transactions; IEEE 
Transactions; Corporate Computing; CIO Journal; CIO 
Magazine. All but the last three journals were added to the 
list of IS core journals. The last three were mentioned by 
only one staff member. 
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After reviewing the responses to the test survey, the 
decision was made to select FORM 2 for use in the actual 
survey. It seemed to elicit an overall positive response 
from the MIS staff. Besides the addition of four more 
journals to the list, minor adjustments were made to the 
cover letter to make it more formal and suitable for a 
general mailing. 

Four hundred copies of the cover letter (Appendix C) 
were made using letterhead to make the survey appear 
official and 400 sheets of blue pastel paper were used for 
the new list of IS core journals. 


B. THE SAMPLE 

In order to get a representative sample of "MIS 
knowledgeable" people who would be generally active and 
experienced in the world of academia, a random sample was 
drawn from the 1992 McGraw-Hill Directory of Information 
Systems Faculty. Four hundred MIS faculty names (out of 
1800) were chosen to represent about 20 percent of the total 
population. To achieve a random sampling, the fourth name 
and fifth alternately were selected from the MIS directory. 
The starting point was the seventh name (which was chosen by 
tossing dice). 

The respondent's and non-respondent's prestige of 
university, region, academic specialty, degree type, and 
degree date would be used in the analysis of our data. 
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Therefore, the names of all chosen to receive surveys were 
entered in a database which contained addresses, 
universities, regions, academic specialties, degree types, 
and degree dates. 

C. CONDUCT OP STUDY 

To obtain the maximum results for a weighted ranking IS 
core journals, a decision was made to send surveys to MIS 
faculty members of universities throughout the country. 
Prestige, as defined earlier, is an abstract, subjective 
quality used in the evaluation of people's work. There is 
no concrete measure of prestige but it does play an 
important role in decision processes. A subjective prestige 
ranking of journals would be the best guide for faculty 
making hiring, promotion, and tenure decisions. 

The 400 surveys were mailed on 10 April, 1993, early 
enough in the spring quarter to arrive before faculty began 
their summer break. Completed forms began to arrive within 
one week. Responses continued to trickle in for three more 
weeks, then there was a serious drop in returns and only ten 
more arrived over the next six weeks. Of the 400 
questionnaires mailed out, nine were returned with 
absolutely no weights given and various excuses for the lack 
of response (the most frequent being that the respondent was 
not familiar with any journals or with MISQ) and four more 
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were returned by the postal service and marked as 
undeliverable. 

One hundred sixty-eight were returned with IS journals 
weighted. Therefore, 396 usable surveys were sent and 42 
percent were returned. This is comparable to other surveys 
which had response rates of 37.8 percent (Hamilton and Ives, 
1983), 31 percent (Doke and Luke, 1987) and almost 50 
percent (Glenn, 1971). 

D. VALIDATION 

The complete database was printed out and validated 
manually by a two-person check. We compared the results of 
each respondent's completed survey to their individual entry 
in the database. One numeric typing error was found and was 
changed before the data was analyzed. 

E. ANALYSIS OF RESPONDENTS VERSUS NON-RESPONDENTS 

An analysis was conducted based on a statistical 
evaluation of the survey data using SPSS release 4.1 for IBM 
VM/CMS. In addition to our overall IS journal weighted 
ranking, we conducted analyses on survey respondents versus 
non-respondents over several dimensions. These include 
prestige of university, region, academic specialty, degree 
type, and degree date. We were interested in studying 
whether any of the variables would affect response rate. 
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1. Prestige of University vs. Response Rates 


Table 1 shows the number of respondents and non¬ 
respondents and whether they obtained their degree from a 
nationally ranked university (Lord, 1993). 


TABLE 1 

RESPONDENTS AND NON-RESPONDENTS BY PRESTIGE 07 UNIVERSITY 


Respond 

Ranked 

Not Ranked 

Total 

No 

95 

138 

233 

Yes 

67 

101 

168 

total 

162 

239 

401 


Ho: No difference in the responses of graduates of 
ranked universities and non-ranked universities. 

Ha: Whether a person responded to the survey and the 
ranking of the university where they received their degree 
are statistically dependent. 


Chi square 

Df 

Significance 

Phi 

.0322 

1 

.858 

.009 


In this case we fail to reject the null hypothesis. 
We conclude that survey response and the ranking of a 
university where a respondent received his or her degree are 
statistically independent. 

2. Regions vs. Response Rates 

Region is defined as one of four geographical areas 
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into which the United States was split. Table 1 gives the 
breakdown of states into their respective region. 


TABLE 2 

REGIONS OF TEE UNITED STATES 


| East 

Midwest 

South 

West 


CT 

DC 

DE 

IA 

IL 

IN 

AL 

AR 

FL 

AK 

AZ 

CA 

MA 

MD 

ME 

MI 

MN 

MO 

GA 

KY 

LA 

CO 

HI 

ID 

NH 

NJ 

NY 

ND 

NE 

OH 

MS 

NC 

SC 

MT 

NM 

NV 

PA 

RI 

VA 

OK 

SD 

WI 

TN 

TX 

WV 

OR 

UT 

WA 

VT 









WY 




The geographical distributions of respondents and 
non-respondents can be seen in Table 2. 


TABLE 3 

RESPONDENTS AND NON-RESPONDENTS BY REGION 


Respond 

East 

Midwest 

South 

West 

Total 

No 

56 

70 

57 

49 

232 

Yes 

40 

43 

53 

32 

168 

Total 

96 

113 

110 

81 

400 


Ho: No difference in the responses according to the 
region of residency. 

Ha: Whether a person responds to the survey and what 
region of the country they are from are statistically 
dependent. 
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In this case we fail to reject the null hypothesis. 
We conclude that survey response rate and what region the 
respondent is from are statistically independent of each 
other. 

The geographical information shows that the sample 
is evenly distributed across the four regions of the United 
States. 

3. Academic Specialties vs. Response Rates 

Table 4 shows the number of respondents and non- 
respondents and their academic specialties (Appendix D). 

The respondent's academic specialty is the area in which 
they work as delineated in the 1992 McGraw-Hill Directory of 
Information Systems Faculty. 


TABLE 4 

RESPONDENTS AND NON-RESPONDENTS BY ACADEMIC SPECIALTY 


Respond 

8.1 

00 

to 

8.3 

8.4 

8.5 

Unk 

Total 

No 

84 

66 

53 

4 

3 

23 

233 

Yes 

59 

53 

46 

3 

1 

6 

168 

Total 

143 

119 

99 

7 

4 

29 

401 


Ho: No difference between responses to the survey 
according to academic specialty. 
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Ha: Whether a person responds to the survey and their 


academic specialty are statistically dependent. 


Chi Square 

Df 

Significance 

Cramer's V § 

7.043 

5 

.217 

.133 1 


In this case we fail to reject the null hypothesis. 
We conclude that survey response rate and academic specialty 
are statistically independent of each other. 

4. Degree Types vs. Response Rates 

The degree types were split into four disciplines; 
IS, business, social sciences, and mathematics and physical 
sciences. The following table shows the breakdown of degree 
types. 


TABLE 5 
DEGREE TYPES 


Information 

Systems 

Business 

Social 

Sciences 

Math & Phys 
Sciences 

Sys Design 
Decision Sci 
Micro App 

MIS and IS 

Art Intel 

Admin 

Statistics 

Accounting 

Ops Research 

Management 

Finance 

Education 

Org Behavior 

Economics 

Psychology 

Sociology 

Philosophy 

Engineering 
System Sci 

Comm Sci 


Table 6 shows the degree type and the number of 
respondents and non-respondents. The four unknowns are 
people whose names and university were in the MIS directory 
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but degree type was not listed, and one anonymous respondent 
who sent in a weighting of the journals. An extra record 
with no name or biographical information (only the 
weightings) was created for the anonymous respondent. 
Therefore, the sample reads 401, vice 400. 


TABLE 6 

RESPONDENTS AND NON-RESPONDENTS BY DEGREE TYPE 


Respond 

IS 

Business 

Soc Sci 

Math & 
Phys Sci 

Unknown 

Total 

No 

95 

55 

43 

21 

19 

233 

Yes 

75 

57 

17 

15 

4 

168 

Total 

170 

112 

60 

36 

23 

401 


Ho: No difference in response to survey according to 
degree type. 


Ha: Whether a person responds to the survey and their 
degree type are statistically dependent. 


Chi Square 

Df 

Significance 

- - ...- 

Cramer's V 

14.277 

4 

.006 

.189 


In this case we reject the null hypothesis. 
Therefore, we can conclude that a relationship exists 
between the survey response rate and the respondent's degree 
type. 

5. Degree Dates vs. Response Rates 

Table 7 shows the number of respondents and non¬ 
respondents and their degree dates (by the decade). The 
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e MIS directory. The 




Ho: No difference in response to the survey according to 
degree date. 

Ha: Whether a person responds to the survey and the date 
they received their degree are statistically dependent. 


Chi Square 

Df 

Significance 

Cramer's V 

2.437 

4 

.656 

.080 


In this case we fail to reject the null hypothesis. 


We conclude that survey response rate and degree date are 
statistically independent of each other. 

Except for type of degree, we found no statistically 
significant differences between respondents and non¬ 
respondents. Therefore, the respondents to this survey may 
be characterized as representative of the population of MIS 
academicians as they are characterized by a random sample. 
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IV. FINDINGS 


A. IS JOURNAL WEIGHTED RANKING 

As with Glenn's (1971) study, the IS journals were 
weighted and ranked by prestige according to the 
respondent's perceptions (Table 8). Because the weights 
assigned to the IS journals were identified as interval 
level data, the mean and standard deviation were identified 
as the appropriate measure of central tendency and 
dispersion, respectively. 

1. Intensity end Extensity of Prestige 

The overall IS journal ranking was also used to 
study the differences between the intensity and extensity of 
prestige, Glenn (1971) states, "the mean weight is 
essentially an indicator of the intensity of prestige, 
whereas the number of respondents who assigned weights 
(Table 8, N column) is a rough indicator of how well the 
journal was known among the respondents, or the extensity of 
the journal's prestige." The number of respondents who 
assigned weights can not be considered a precise measure of 
the extensity of prestige, since some respondents may have 
thought a journal prestigious but didn't feel qualified to 
make a precise judgement. 

We tested the precision of our measure of extensity. 
We formulated a figure of total prestige by multiplying the 
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intensity by the extensity (Table 8, Total column) to see if 
the order of the IS journal ranking would be affected. 

As can be seen in Table 8, ORSA Journal on Computing 
received a higher mean weight (9.2) than Data Base (6.88) 
but the latter was known and evaluated by more respondents. 
The journals can be ordered by either criteria (mean weight 
or number of respondents who assigned weights) but there is 
no sound basis to conclude that neither ORSA Journal on 
Computing or Data Base had more overall prestige. On the 
other hand, if two of the IS journals had similar mean 
weights, than total prestige could be determined. For 
instance. Information and Management (7.53) and Omega (7.5) 
had similar mean weights. The former was rated by 128 
respondents and the latter by 80—an indication of the 
extensity of the prestige of Information and Management. 

Table 8 compares the weighted rank order of IS 
Journals (Mean column) and the rank order when the weighted 
ranking is multiplied by the N of respondents (Order 
column). A Spearman's Rank order correlation shows a 
moderately strong association between the two rank orders 
(rho = .76). We therefore concluded that the weighted rank 
order was not a biased indicator of IS journal prestige when 
accounting for extensity. 
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TABLE 8 


OVERALL 18 JOURNAL WEIGHTING 




N 

SMDav 

ToW 

(Mar 

Management Sdmoi 

12.67 

141 

6.01 

1772.37 

1 

Information Systems Rnssth 

11 A3 

124 

446 

.1417.32 

5 

CcmmunteaOcn* ol Oia ACM 

11.06 

162 

461 

1886.68 

2 

EEE Transactions 

10.88 

122 

6.64 

1327.36 

7 

AdmMOT** Sclsno* Quarterly 

10.56 

101 

6.78 

1068.58 

14 

ACM Transaction* 

10.1 

120 

6.06 

1302.0 

8 

Management Information Systems Qusrterty 

10.01 

154 

1.08 

1641.54 

3 

ACM Computing Survay* 

6j67 

126 

6.41 

1375.88 

6 

nwvu Dimness nwNw 

0.8 

147 

6.16 

1466.3 

4 

Acadamy at Managamant Journal 

0.62 

113 

447 

1087.06 

13 

ORSA Journal on Computing 

0.2 

74 

6.07 

080.8 

22 

Dsctaion Sctencos 

0.14 

137 

6.11 

1262.18 

10 

IEEE Computer 

0.04 

123 

4.22 

1111.82 

12 

Journal tf Managamant HormaOon Systems 

8.08 

146 

3.06 

1302.1 

8 

Sloan Managamant Rsvtear 

6.0 

133 

4.32 

1183.7 

11 

Accounting Ravtew 

IM 

06 

6.18 

737.88 

20 

Dsdslon Stgtport Systems 

6.4 

114 

4.62 

057.6 

17 

tntortooot 

7.81 

127 

438 

100457 

16 

Information and Managamant 

7.63 

128 

3.02 

863.84 

16 

Omaga 

7.6 

80 

2.87 

600 

24 

DaiaBasa 

6J8 

136 

3.64 

828.8 

18 

Journal ol Systems Managamant 

6.66 

127 

3.4 

833.12 

10 

EOP Analyzar 

S M 

104 

6.7 

612.56 

23 

Datamation 

487 

140 

4.66 

725.63 

21 


B. PRESTIGE OF UNIVERSITY AND JOURNAL WEIGHTING 

We examined the relationship between university ranking 
and journal weighting. The variable, university ranking, 
has two possible categories ("ranked" or "not ranked"). 
Table 9 shows the list of IS journals in order of the 
"ranked" column. Each column displays the order placement 
number and mean weight (in parenthesis) for each journal. 
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TABLE 9 


PRE8TIGE OF UNIVERSITY 


Journal 

Ranked 

Not Ranked 

Management Science 

1 (14.01) 

1 (11.62) 

Information Systems Research 

2(12.78) 

5(10.55) 

Communications of the ACM 

3(11.51) 

3(10.79) 

IEEE Transactions 

4(11.27) 

4(10.58) 

Administrative Science Quarterly 

4(11.27) 

6(10.15) 

ACM Transactions 

5 (10.80) 

9 (9.61) 

ACM Computing Surveys 

6(10.75) 

10 (9.45) 

ORSA Journal on Computing 

7(10.69) 

16 (8.24) 

Harvard Business Review 

8(10.31) 

8 (9.64) 

Management Information Systems Quarterly 

9(10.96) 

2(10.93) 

Decision Sciences 

10(10.13 

14 (8.52) 

IEEE Computer 

11 (9.64) 

12(8.61) 

Sloan Management Review 

12 (9.56) 

15(8.46) 

Academy of Management Journal 

13(9.46) 

7(9.71) 

Journal of Management Information Systems 

14 (9.27) 

11 (8.80) 

Decision Support Systems 

15(9.16) 

18 (7.82) 

Accounting Review 

16 (8.61) 

13 (8.57) 

Omega 

17 (8.08) 

20(7.09) 

Interfaces 

18(7.99) 

17(7.86) 

Information and Management 

19 (7.27) 

19 (7.70) 

Journal of Systems Management 

20 (6.53) 

22 (6.58) 

DataBase 

21 (6.17) 

21 (6.95) 

EDP Analyzer 

22 (5.39) 

23 (6.22) 

Datamation 

23 (4.32) 

24 (5.23) 


By using university rank as the independent variable, we 
attempted to determine whether graduating from one of the 
prestigiously ranked universities would have an effect on 
how respondents weighted IS journals (Lord, 1993). The 
TTEST was selected to evaluate this relationship since there 
were only two categories. 

The null hypothesis is that there is no difference in 
the weighted means of each IS journal and prestige of 
university. The alternative hypothesis is that there is a 
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statistically significant difference in the weighted means 
of each IS journal and prestige of university. 


TABLE 10 

PRESTIGE OF UNIVERSITY AND JOURNAL WEIGHTING (TTE8T) 


Journal 

F Value 

2-Tali Prob 

Academy of Management Journal 

1.140 

0.616 

Accounting Review 

1.110 

0.725 

ACM Computing Surveys 

1.090 

0.748 

ACM Transactions 

1.480 

0.120 

Administrative Science Quarterly 

1.470 

0.174 

Communications of the ACM 

1.140 

0.559 

DataBase 

1.050 

0.858 

Datamation 

1.470 

0.119 

Decision Sciences 

1.530 

0.081 

Decision Support Systems 

1.540 

0.108 

EDP Analyzer 

3.280 

* 0.0 

Harvard Business Review 

1.190 

0.458 

IEEE Computer 

2.740 

*0.0 

IEEE Transactions 

1.470 

0.149 

Information and Management 

1.480 

0.146 

Information Systems Research 

1.950 

*0.01 

Interfaces 

2.440 

*0.001 

Journal of Management Information Systems 

1.580 

0.056 

Journal of Systems Management 

2.770 

•0.0 

Management Information Systems Quarterly 

1.920 

*0.005 

Management Science 

1.840 

*0.0111 

Omega 

2.280 

*0.01 

ORSA Journal on Computing 

1.060 

0.879 

Sloan Management Review 

1.250 

0.395 


We rejected the null hypothesis for the starred journals 
in Table 10 with a statistical significance (alpha) of .05 
or less. We failed to reject the null hypothesis for non- 
starred journals. Overall, in sixteen out of twenty-four 
journals, university ranking did not have a statistically 
significant relationship with journal weightings. 
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C. REGION AND JOURNAL WEIGHTING 


We examined the relationship between region and journal 
weighting. The variable, region, has four possible values 
(East, Midwest, South, and West). Table 11 shows the list 
of IS journals in order of the "East" column. All columns 
display the rank order placement number and mean weight (in 
parenthesis) for each journal. 


TABLE 11 
REGIONS 


Journal 

IEEE Transactions 

Management Science 

Communications of the ACM 

Information Systems Research 

Administrative Science Quarterly 

ACM Transactions 

ORSA Journal on Computing 

ACM Computing Surveys 

Management Information Syttems Quarterly 

Harvard Business Review 

Sloan Management Review 

Decision Sciences 

Academy of Management Journal 

IEEE Computer 

Journal of Management Information Systems 
Interfaces 

Decision Support Systems 
Accounting Review 
Omega 

Information and Management 

Journal of Systems Management 

DataBase 

EDP Analyzer 

Datamation 


East 

MdWMt 

South 

West 

1 (13.75) 

7 (10.12) 

6 (9.63) 

6 (10.48) 

2 (13.69) 

t (13.77) 

1 (11.40) 

2(11.38) 

3(12.11) 

5 (10.50) 

3(10.66) 

3 (11.32) 

4(11.77) 

4(11.12) 

2 (11.24) 

1 (11.71) 

5(10.92) 

3(12.04) 

7(9.27) 

11 (10.24) 

6 ( 10 . 81 ) 

8(10.32) 

12 (8.78) 

5 (10.85) 

7(10.26) 

16 (8.35) 

14 (8.46) 

13 (9.88) 

8 (10.09) 

9(9.77) 

10 (9.08) 

4(11.31) 

9(10.03) 

2(12.32) 

5(9.78) 

9 (10.33) 

10 (9.78) 

11 (9.46) 

4(9.84) 

6(10.79) 

11 (9.61) 

15 (8.41) 

16(8.13) 

12(9.96) 

12(9.46) 

12 (9.37) 

13 (8.66) 

15(9.31) 

13 (9.26) 

10 (9.72) 

9 (9.14) 

7 (10.71) 

14 (9.10) 

13 (9.00) 

11 (9.05) 

17(9.00) 

15 (8.71) 

14 (8.92) 

8(9.21) 

16(9.04) 

16 (8.07) 

18(7.27) 

17 (7.69) 

18(8.85) 

17(7.95) 

17 (8.07) 

15 (8.34) 

14(9.50) 

18 (7.58) 

8 (9.96) 

21 (6.80) 

10(10.26) 

19 (7.47) 

18(7.27) 

19 (7.58) 

22 (7.71) 

20(7.23) 

19 (7.13) 

18 (7.60) 

19 (8.54) 

21 (6.34) 

21 (5.74) 

22 (6.79) 

23(7.43) 

22(6.18) 

20 (6.47) 

20(7.24) 

21 (7.76) 

23 (4.52) 

22 (4.92) 

23 (6.37) 

20(8.27) 

24 (3.00) 

23 (4.64) 

24 (5.43) 

24(6.47) 
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Analysis of variance (ANOVA) was selected as the 
appropriate statistical test because the independent 
variable academic region had four possible categories, 
null hypothesis is that there is no difference in the 
weighted means of each IS journal by region. The 
alternative hypothesis is that there is a statistically 
significant difference in the weighted means of each IS 
journal and region. 


TABLE 12 

REGION AND JOURNAL WEIGHTING (ANOVA) 


Journal 

F Value 

Slg. of F 

Academy of Management Journal 

0.403 

0.751 

Accounting Review 

4.657 

•0.007 

ACM Computing Surveys 

2.168 

0.099 

ACM Transactions 

1.260 

0.296 

Administrative Science Quarterly 

1.649 

0.187 

Communications of the ACM 

0.222 

0.881 

Data Base 

1.716 

0.172 

Datamation 

3.818 

*0.013 

Decision Sciences 

1.383 

0.255 

Decision Support Systems 

£175 

0.100 

EDP Analyzer 

3.870 

•0.014 

Harvard Business Review 

1.266 

0.292 

IEEE Computer 

0.863 

0.465 

IEEE Transactions 

1.206 

0.315 

Information and Management 

2.299 

0.084 

Information Systems Research 

0.060 

0.981 

Interfaces 

1.458 

0.233 

Journal of Management Information Systems 

0.332 

0.802 

Journal of Systems Management 

3.628 

•0.017 

Management Information Systems Quarterly 

1.277 

0.288 

Management Science 

1.760 

0.161 

Omega 

0.098 

0.961 

ORSA Journal on Computing 

0.934 

0.433 

Sloan Management Review 

2.186 

0.097 


The 
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We rejected the null hypothesis for the starred journals 


in Table 12 with a statistical significance (alpha) of .05 
or less. We failed to reject the null hypothesis for non- 
starred journals. Overall, region did not have a 
statistically significant relationship to journal weighting. 


D. ACADEMIC SPECIALTY AMD JOURNAL WEIGHTING 

We examined the relationship between academic specialty 
and journal weighting. The variable, academic specialty, 
fell into six nominal categories (8.1, 8.2, 8.3, 8.4, 8.5 
and unknown). Table 13 shows the list of IS journals in 
order of the "8.1" column. All columns display the rank 
order placement number and mean weight (in parenthesis) for 
each journal. 
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TABLE 13 


ACADEKIC SPECIALTY 


Journal 

9.1 

IL2 

U 

u 

84 

Unknown 

Management Sdanoa 

1 (12.86) 

1 (11.66) 

1 (12.70) 

3(8.67) 


2(29.00) 

IEEE Transaction* 

2(12.18) 

11 (8.91) 

4(10.50) 

6(740) 


3(24.50) 

Managamant Information Syatams Quarterly 

3(11.73) 

6 (10.16) 

8(941) 

1 (10.00) 

2(6-00) 

13(1147) 

CommunicaMonaalttiaACM 

4(11.52) 

3(11.09) 

3 (10.73) 

7(7.00) 

1(7-00) 

8 (15.00) 

/VJIMMtfUVf oGWnOI \JUmWTTj 

8(1142) 

4(10.74) 

10(9.40) 

6(8.00) 


7(16.00) 

Information Systems Raaaareh 

6(10.86) 

2(11.27) 

2(12.16) 

11 (5.50) 


7 (16.00) 

ACM Transactions 

7 (10.87) 

12(8.76) 

5(1040) 

6(7.50) 


6 (17.50) 

ORSA Journal on Computing 

8(10.86) 

18 (7.09) 

14(9.04) 

7(7.00) 


1 (30.00) 

ACM Computing Surveys 

8(10.47) 

8(9.34) 

8(948) 

2(9.67) 

3(5.00) 10(1343) 

Aoadamy of Managamant Journal 

10(9.82) 

5(10.41) 

16 (8.69) 

4(8.33) 


19(543) 

Harvard Businass Review 

11 (8.73) 

7(944) 

7(946) 

1 (10.00) 


9 (14.67) 

Decision Sdenoes 

12(9.62) 

15(8.09) 

11 (9.39) 

8(6.67) 


7(16.00) 

IEEE Computer 

13 (8.18) 

14(8.51) 

15(8.95) 

9(840) 

4(4.00) 

4(23.00) 

Journal of Managamant Information Systems 

14 (8.96) 

10(8.96) 

12(9.12) 

10(5.67) 


14(11.00) 

Decision Support Systems 

IS (8.65) 

18(7.32) 

13(9.07) 

12(5.00) 


12 (12.67) 

Sloan Managamant Review 

16 (8.37) 

13 (8.64) 

9(9.66) 

4(8.33) 


11 (13.00) 

Accounting Review 

17(8.10) 

9(9.04) 

18(846) 

3(8.67) 


12 (12.67) 

Interfaces 

18 (7.83) 

16 (7.73) 

17(849) 

14(4.50) 


15 (10.00) 

Omega 

19(7.52) 

17(7.39) 

21 (7.27) 

15 (4.00) 


5(20.00) 

Infomiallon and Management 

20(7.43) 

16(7.73) 

19 (7.58) 

16(3.00) 


17(9.00) 

DataBase 

21 (7.30) 

20 (6.21) 

20(7.35) 

16(3.00) 

1(7.00) 

18(8.00) 

Journal of Systems Management 

22(6.81) 

21 (6.13) 

23 (6.58) 

10(5.67) 


16(9.33) 

EDP Analyzer 

23(4.97) 

22(5.92) 

22(6.83) 

1 (10.00) 

2(6.00) 

21 (3.00) 

Datamation 

24(4.55) 

23(4.67) 

24 (5.67) 

13(4.67) 

2(6.00) 

20(4.00) 

Analysis of variance 

(ANOVA) 

was 

selected as 

the 



appropriate statistical test because the independent 


variable academic specialty had six possible categories. 

The null hypothesis is that there is no difference in the 
weighted means of each IS journal and academic specialty. 
The alternative hypothesis is that there is a statistically 
significant difference in the weighted means of each IS 


journal according to academic specialty. 
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TABLE 14 


ACADEMIC SPECIALTY AMD JOURNAL WEIGHTING (ANOVA) 


Journal 

F Value 

Slg.of F 

Academy of Management Journal 

1.485 

0.234 

Accounting Review 

0.660 

0.522 

ACM Computing Surveys 

0.194 

0.90 

ACM Transactions 

1.065 

0.351 

Administrative Science Quarterly 

1.101 

0.339 

Communications of the ACM 

0.908 

0.441 

DataBase 

1.721 

0.170 

Datamation 

0.422 

0.738 

Decision Sciences 

0.516 

0.599 

Decision Support Systems 

2.280 

0.111 

EDP Analyzer 

0.239 

0.869 

Harvard Business Review 

0.006 

0.994 

IEEE Computer 

1.168 

0.329 

IEEE Transactions 

1.580 

0.214 

Information and Management 

3.228 

*0.045 

Information Systems Research 

3.077 

3.077 

Interfaces 

1.185 

0.312 

Journal of Management Information Systems 

1.304 

0.277 

Journal of Systems Management 

0.237 

0.790 

Management Information Systems Quarterly 

4.389 

*0.006 

Management Science 

0.455 

0.636 

Omega 

1.619 

0.209 

ORSA Journal on Computing 

0.739 

0.484 

Sloan Management Review 

0.582 

0.561 


We rejected the null hypothesis for the starred journals 
in Table 14 with a statistical significance (alpha) of .05 
or less. We failed to reject the null hypothesis for non- 
starred journals. Overall, academic specialty did not have 
a statistically significant relationship in journal 
weighting. 


E. DEGREE TYPE AND JOURNAL WEIGHTING 

We examined the relationship between degree type and 
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journal weighting. The variable, degree type has five 
possible values (Information Systems, Business, Social 
Sciences, Mathematic and Physical Sciences and Unknown). 
Table 15 shows the list of IS journals in order of the M IS" 
column. All columns display the rank order placement number 
and mean weight (in parenthesis) for each journal. 


TABLE 15 
DEGREE TYPE 
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Analysis of variance (ANOVA) was selected as the 
appropriate statistical test because the independent 
variable degree type had five possible categories. The null 
hypothesis is that there is no difference in the weighted 
means of each IS journal and degree type. The alternative 
hypothesis is that there is a statistically significant 
difference in the weighted means of each IS journal and 
degree type. 


TABLE 16 

DEGREE TYPE AMD JOURNAL WEIGHTING (ANOVA) 


Journal 

F Value 

Slg. of F 

Academy of Management Journal 

2.305 

0.086 

Accounting Review 

2.020 

0.126 

ACM Computing Surveys 

0.324 

0.808 

ACM Transactions 

0.009 

0.999 

Administrative Science Quarterly 

0.668 

0.575 

Communications of the ACM 

0.955 

0.418 

Data Base 

2.010 

0.120 

Datamation 

1.449 

0.235 

Decision Sciences 

0.926 

0.433 

Decision Support Systems 

1.833 

0.150 

EDP Analyzer 

1.864 

0.146 

Harvard Business Review 

0.696 

0.557 

IEEE Computer 

1.115 

0.349 

IEEE Transactions 

0.306 

0.821 

Information and Management 

2.942 

•0.039 

Information Systems Research 

0.692 

0.560 

Interfaces 

1.250 

0.298 

Journal of Management Information Systems 

0.095 

0.962 

Journal of Systems Management 

3.159 

•0.03 

Management Information Systems Quarterly 

2.499 

0.065 

Management Science 

1.042 

0.379 

Omega 

0.159 

0.923 

ORSA Journal on Computing 

0.929 

0.436 

Sloan Management Review 

3.597 

* 0.017 
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We rejected the null hypothesis for the starred journals 
in Table 16 with a statistical significance (alpha) of .05 
or less. We failed to reject the null hypothesis for non¬ 
star red journals. Overall, degree type did not have a 
statistically significant relationship in journal weighting. 

F. DEGREE DATE AND JOURNAL WEIGHTING 

We examined the relationship between degree date and 
journal weighting. The variable, degree date has five 
possible values (1950's, 1960's, 1970's, 1980's, and 
1990's). Table 17 shows the list of IS journals in order of 
the "1950's" column. All columns display the rank order 
placement number and mean weight (in parenthesis) for each 
journal. 
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TABLE 1? 


DEGREE DATE 


Journal 

taco 

1880 

1870 

1880 

1888 

Information Syitama Raaoarch 

1 (1223) 

2(12.53) 

3(11.06) 

3(1028) 

3(1221) 

Mana 0 *m*nt Sdmoc 

2(11,83) 

102-64) 

1 (12.61) 

1(1228) 

1(13.70) 

ACM Computing Sunray* 

3(10.13) 

12(8.70) 

6(1023) 

10(8.46) 

6(10.73) 

Managamant Information Syatama Quarterly 

4 (1020) 

10 0021 ) 

8(1024) 

7(8.81) 

11 ( 1020 ) 

ACM Transactions 

S (9.88) 

7 (10.80) 

11 (8.63) 

6 (10.04) 

7(11.13) 

IEEE Transaction* 

6(9.33) 

3(12.00) 

201-23) 

0(10.03) 

4(1128) 

Communication* of tiw ACM 

7(8.76) 

5 (11 JO) 

6(1028) 

2 ( 10 . 88 ) 

20223) 

Harvard Buolnaa* RavWw 

6 (8.33) 

8 ( 10 . 66 ) 

10(824) 

8(8.81) 

601-07) 

Journal of Man^jamant Information fiyilrm 

8(7.78) 

15(820) 

13(828) 

18(828) 

12(824) 

Aoadomy of Managomant Journal 

10 (7.71) 

16(7.81) 

7(10.18) 

8(8.67) 

6(1123) 

Omaga 

11(7.80) 

17 (725) 

21(723) 

20(728) 

18(725) 

Inforfacts 

12(7.20) 

22(5.63) 

17(8.30) 

18(823) 

17(8.75) 

IEEE Computor 

13(7.00) 

6 (10.74) 

18(8.68) 

14(8.80) 

16(9.44) 

Admlnlatrtflva Sdanca Quarterly 

14(8.71) 

4(11.50) 

4(10.80) 

4(1022) 

6(1124) 

Sloan Managamant Ravtaw 

16(8.43) 

13(8.64) 

14(8.08) 

16(8.56) 

10(10.07) 

Oadslon Sdanoaa 

1ti(528) 

11 (921) 

12(823) 

11(828) 

18(821) 

Information and Managomant 

17(5.71) 

19(6.88) 

18(8.15) 

18(7.47) 

18(7.73) 

Journal of Syatems Managomant 

1ti(5S0) 

20(8.47) 

22(728) 

22(028) 

21 ( 8 . 00 ) 

Data Basa 

18(528) 

21 ( 8 . 00 ) 

18 (828) 

21 (823) 

20(7.12) 

ORSA Journal or Computing 

20(5.00) 

14(8-46) 

8 ( 8 . 86 ) 

12 (8.97) 

14(8.50) 

Accounting Ravtaw 

21(4.80) 

8 (10.53) 

20(7.71) 

13(8.82) 

15(924) 

EDP Analyzar 

22(4.25) 

24(420) 

23(7.08) 

23(6.86) 

22(520) 

Dadsion Support Systems 

23(4.14) 

18(8.88) 

15(8.78) 

17(820) 

13(823) 

Datamation 

24(3.44) 

23(462) 

24(6.33) 

24(4.61) 

23(484) 


Analysis of variance (ANOVA) was selected as the 
appropriate statistical test because the independent 
variable degree date had five possible categories. The null 
hypothesis is that there is no difference in the weighted 
means of each IS journal and degree date. The alternative 
hypothesis is that there is a statistically significant 
difference in the weighted means of each IS journal and 
degree date. 
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TABLE IS 


DEGREE DATE AMD JOURNAL VEIGHTIMG (AMOVA) 


Journal 

F Value 

«g.of F 

Acadamy of Managamant Journal 

1.688 

0.156 

Accounting RaWew 

1.582 

0.184 

ACM Computing Survey* 

0.317 

0.866 

ACM Transactions 

0.324 

0.661 

Administrates Solanos Quarterly 

1.011 

0.406 

Communications oflha ACM 

1.259 

0.289 

Data Bass 

2.474 

* 0.048 

Datamation 

0.335 

0.884 

Dsdsion Sdsncss 

0.856 

0.434 

Dsdslon Support Systsms 

1.835 

0.127 

EDP Analyzer 

0.866 

0.487 

Harvard Businsss Review 

0.513 

0.726 

IEEE Computer 

1.320 

0.267 

IEEE Transactions 

0.529 

0.714 

Information and Managemsnt 

1.242 

0.297 

Information Systsms Research 

0.693 

0.598 

Interfaces 

1.570 

0.187 

Journal of Management Information Systsms 

1.332 

0.261 

Journal of Systems Managamant 

0.903 

0.464 

Management Information Systems Quarterly 

0.341 

0.850 

Management Science 

0.190 

0.943 

Omega 

0.061 

0.993 

ORSA Journal on Computing 

0.443 

0.777 

Sloan Management Review 

1.046 

0.386 


We rejected the null hypothesis for the starred journals 
in Table 18 with a statistical significance (alpha) of .05 
or less. We failed to reject the null hypothesis for non- 
starred journals. Overall, degree date did not have a 
statistically significant relationship in journal weighting. 

In summary, none of the independent variables (prestige 
of university, region, academic specialty, degree type, or 
degree date) had an overall statistically significant 
relationship with the journal weightings. 














V. DISCUSSION AND ANALYSIS 


A. COMPARISON OP RANKINGS 

Table 19 gives the results of six studies in which IS 
journals were ranked. These studies were conducted from 
1983 to 1993. There is a noticeable similarity in the 
rankings. All of the rankings have at least five of the top 
ten journals in common, although not in the same order. 
Management Science, MISQ, and Communications of the ACM were 
among the top ten journals in all six studies. Harvard 
Business Review, Computing Surveys and Data Base were 
present in five and four, respectively, of the six rankings. 
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TABLE 19 


THE TOP TEN JOURNALS AS COMPARED IN SIX STUDIES 


Doke & 
Luke 

Vogel & 
Wetherbe 

Hamilton 
Ives & 

Shim et 
al. 

Ramesh 6 
Stohr 

Bayes & 
Huskey 

MISQ 

Comms of 
the ACM 

MISQ 

Mgmt 

Science 

Mgmt 

Science 

Mgmt 

Science 

J of MIS 

Harvard 

Bus Rev 

EDP 

Analyzer 

Comms of 
the ACM 

Ops 

Research 

IS 

Research 

Comms of 
the ACM 

MISQ 

Data 

Base 

MISQ 

Comms of 
the ACM 

Comms of 
the ACM 

Decision 

Sciences 

Info & 

Mgmt 

Mgmt 

Science 

Harvard 

Bus Rev 

J of the 
ACM 

IEEE 

Trans 

Mgmt 

Science 

Sloan 

Mgmt Rev 

Data¬ 

mation 

Decision 

Sciences 

ACM Trans 
Database 

Admin 

Science 

Qtrly 

Data¬ 

mation 

Mgmt 

Science 

Harvard 

Bus Rev 

Sloan 

Mgmt Rev 

IEEE 

Trans 

S/W Eng 

ACM 

Trans 

Harvard 
Bus Rev 

J of Data 
Education 

Computing 

Surveys 


Art Intel 

MISQ 

J of 

Comp IS 

Data¬ 

mation 

Comms of 
the ACM 

Computing 

Surveys 

Cognitive 

Science 

ACM Comp 
Surveys 

J of IS 
Mgmt 

Data Base 

Trans on 
Dbase Sys 

IBM Sys 
Journal 

MISQ 

Harvard 
Bus Rev 

Data 

Base 

Trans on 
Dbase Sys 

Info and 
Mgmt 

Data Base 

Computing 

Surveys 

Acad of 
Mgmt J 
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B. INTRODUCTION OF HEN JOURNALS 


Some of the variations in the top ten rankings may be 
attributed to the introduction of new journals. Not only 
are new journals created but some journals cease to be 
published. Exact beginning or ending publication dates of 
the journals listed in all six surveys were not researched, 
but at least one new journal. Information Systems Research 
was included in the last two studies (Ramesh and Stohr, 

1987, and Hayes and Huskey, 1993). Journals need time to 
build a reputation, Information Systems Research may not 
have been established enough to make the top ten journal 
list in Ramesh and Stohr's study, but it did in our study. 
This explains why earlier studies would have similar core 
lists of journals, they may have used a smaller list due to 
the lack of availability of newly published IS journals. It 
was also noted that EDP Analyzer had changed its name to A DP 
Analyzer. 


C. SAMPLE DIFFERENCES 

In addition to the introduction of new journals, 
differences in the samples taken by each study could account 
for the variation in ranking. Doke and Luke (1987) surveyed 
deans of business schools (with instructions to distribute 
them to professors, associate professors and assistant 
professors). How the deans chose to distribute the surveys 
among the faculty is unknown. Doke and Luke (1987) sent 
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their surveys primarily to business schools, whereas the 
surveys in our study went directly to MIS faculty . 

Hamilton and Ives (1983) sent surveys to "knowledgeable 
recognized experts." Shim et al. (1987) used only 
"outstanding" senior researchers for their survey. They 
excluded any junior faculty or lesser known senior faculty 
opinions, even though the latter may be the decision makers 
in hiring, tenure or promotion situations. 

The survey in our study validates the findings of 
previous studies of IS journal rankings. MIS academicians 
have not significantly shifted their subjective prestige 
ranking of IS journals in the last decade. The results show 
that half of the journals thought of as prestigious in five 
other surveys continue to be thought of highly in our 
survey. 

D. VALUE OP SURVEY METHODOLOGY 

The main value of this study is the use of prestige 
weights rather than ranking. Prestige weights allow 
decision makers to quantify the differential prestige of 
journals in hiring, promotion or tenure decisions. If a 
candidate for promotion has articles in different journals, 
the mu frequencies of the articles in journals can be 
multiplied by the prestige weight to obtain a more 
meaningful indicator of the value of a publishing record. A 
ranking of journals would not have given any distance type 
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measure between the journals, only the order in which they 
are perceived. A weighting gives both a distance between 
journals in addition to order of preference. 

The weighting of IS journals can also be valuable to 
those who are evaluating prestige and productivity of IS 
departments. (Allen, 1993) 

E. SURVEY DIFFERENCES 

Several rankings of IS journals have been conducted 
(Hamilton and Ives, 1983; Doke and Luke, 1987; Vogel and 
Wetherbe, 1984). Our study used a different survey 
methodology. We used a weighting to develop a ranking. As 
discussed previously, the weighting can provide a tool for 
decision makers to use. 

We used a random sample of MIS professionals from the 
1992 McGraw-Hill Directory of Management Information Systems 
Faculty. Ramesh and Stohr (1987) also used a random sample 
from the MIS directory, but limited their sample to CS 
professionals. We used a larger sample size (400) compared 
to the 200 used by Ramesh and Stohr (1987). The sample of 
400 in this study was not limited to one segment of the MIS 
community, but included faculty members from all academic 
ranks. 

Our study differed from earlier studies in that it 
compared respondents to non-respondents in regard to 
prestige of university, region, academic specialty, degree 
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type, and degree date. We determined if any of the 
independent variables were statistically significant between 
respondents anl non-respondents. The database allowed us to 
compare weights calculated by cross-referencing respondents 
with certain variables. As far as we know, this has not 
been done in any previous studies. As seen by our results 
in Tables 8 through 18, there was no significant difference 
in the ranking derived from the mean weights between the 
different categories of each variable. 

The comparison of independent variables showed no 
significant differences in IS journal weightings. The 
respondent's prestige of university, region, academic 
specialty, degree type, or degree date had no significant 
impact on how the journals were weighted. This proves that 
our random sample represented an even population 
distribution between all the categories of the independent 
variables. 

F. LESSONS LEARNED 

There are several lessons that were learned during the 
survey process. But there were three aspects of our study, 
had we the opportunity, would do differently. 

One of the studies (Hawkins et al., 1973) of economic 
journals used an interesting tactic. The inclusion of 
fictitious journals in their core list of journals proved 
that respondents do recognize general academic journals more 
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than specialized academic journals. It was, however, 
surprising that one of the fictitious journals (J.E.S.T.) 
was recognized and ranked in the top third of their list of 
economic journals. 

Although we accepted suggestions for additional journals 
to the core list of IS journals from respondents in the test 
survey, we did not leave an open-ended opportunity for 
respondents in the actual survey. A more complete and 
diverse list of IS journals could have been compiled and we 
would have avoided any bias created by limiting the list of 
core journals. 

If we had started the survey process earlier, a follow 
up survey could have been done. Even a reminder (with the 
original survey) could have been sent to those who had not 
responded after a set period of time and increased our 
response rate. 

G. CONCLUSION 

The weighted ranking of IS journals could be used by MIS 
faculty when making decisions in regards to hiring, 
promotion, and tenure. This was accomplished by using a 
survey asking respondents to weight journals according to 
their perceived prestige value. A ranking was formed based 
on the mean weights of the IS journals. Statistical 
analysis showed that independent variables did not have a 
significant impact on the perceived prestige of journals. 
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Professionals can use this study for exercises in 
performance evaluation of research organizations. We have 
researched several methodologies in other disciplines for 
ranking journals. Our research methodology uses aspects of 
these different methodologies and can provide a tool for 
measuring performance evaluation. 

Currently DoD is trying to use the CIM initiative and 
business re-engineering to improve business practices. 

These two processes are intertwined to evaluate performance 
and make it more efficient. Our study is of particular 
interest to performance evaluations in the context of DoD's 
policy of re-engineering business processes. The research 
methodology explored techniques to quantify an 
impressionistic rating. This would be useful in quantifying 
ratings of performance evaluations in management of DoD 
laboratories and schools. 
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APPENDIX A 


Dear Colleague: 

We are sending out this survey as part of a thesis project that 
will attempt to rank the Information System core journals in order 
to show which publications are held in the highest esteem by 
Management Information System faculty members. This should be of 
great value to researchers looking to publish in order to get 
tenure. 

The following list of Information Systems publications is 
provided for you to rank (one through ten) in accordance with your 
judgement of the importance of their contributions to the field. 
Of course, the importance of the individual publications in certain 
categories may vary greatly, but think in terms of their overall 
importance. 

Please put the rank number in the space provided after the 
publication title. 

After completing the form, please return it via the self- 
addressed stamped envelope. Please indicate if you would like a 
copy of the results. 

We sincerely thank you for all your assistance in this project. 
If you have any questions please call us at (408) 656-7124. 
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Name_ 

JOURNAL 

Academy of Management Journal 

Accounting Review 

ACM Computing Surveys 

Administrative Science Quarterly 

Communications of the ACM 

Data Base 

Datamation 

Decision Sciences 

EDP Analyzer 

Harvard Business Review 

IEEE Computer 

Information and Management 

Information Systems Research 

Interfaces 

Journal of Management Information Systems 
Journal of Systems Management 
Management Information Systems Quarterly 
Management Science 
Omega 

Sloan Management Review 


RANKING (top ten only) 
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APPENDIX B 


V 


Dear Colleague: 

We are sending out this survey as part of a thesis project that 
will attempt to rank the Information System core journals in order 
to show which publications are held in the highest esteem by 
Management Information System faculty members. This should be of 
great value to researchers looking to publish in order to get 
tenure. 

On the form please assign weights to the following kinds of 
information system publications according to your judgment of their 
prestige in the field. Of course, the importance of individual 
publications in each category varies greatly, but please think in 
terms of their average importance. Use the Management Information 
Systems Quarterly (MISQ) as your standard of reference. A weight 
of 10 has been arbitrarily assigned to MISQ, so that a publication 
only half as important as MISQ should be assigned a weight of 5, a 
publication twice as important as MISQ should be assigned a weight 
of 20, and so forth. Negative values are acceptable. 

If you do not know enough about a journal to assign a weight to 
its articles, please place an X in the space provided for the 
weight. 

Remember, a weight of 10 has arbitrarily been assigned to 
articles in the MISQ. Use this as your standard for assigning 
other weights. 

After completing the form, please return it via the self- 
addressed stamped envelope. Please indicate if you would like a 
copy of the results. 

We sincerely thank you 'or all your assistance in this project. 
If you have any questions please call us at (408) 656-7124. 
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JOURNAL 


WEIGHT 


Academy of Management Journal 

Accounting Review 

ACM Computing Surveys * 

Administrative Science Quarterly 

Communications of the ACM 

Data Base 

Datamation 

Decision Sciences 

EDP Analyzer ** 

Harvard Business Review 
IEEE Computer *** 

Information and Management 
Information Systems Research 
Interfaces 

Journal of Management Information Systems 
Journal of Systems Management 
Management Information Systems Quarterly 
Management Science 
Omega 

Sloan Management Review 


10 


* ACM: Association for Computing Machinery 

** EDP: Electronic Data Processing 

*** IEEE: Institute of Electronic and Electrical Engineering 







APPENDIX C 



DEPARTMENT OF THE NAVY 

NAVAL POSTGRADUATE SCHOOL 
MONTEREY. CA S3S4>9100 


Dear Colleague, 

I beg your indulgence of an imposition upon your time and patience. I am asking you to 
assign relative weights to a selection of IS journals. 

You may find this exercise to be a bit artificial. I cannot dismiss that assessment. 
Nonetheless, I speculate that many of us form a ranking or rating of IS publications in our 
minds, however impressionistic they may be. We may do it when evaluating a colleague's 
vita, when assessing the credibility of an IS department or when musing about your own 
value in the academic market. 

This is merely an attempt to impose a quantification exercise on what we may already be 
doing. This rating technique has been borrowed from reference disciplines in the social 
sciences. I just want to see if it is applicable within the IS community. 

On the attached sheet, please assign a weight to each journal about which you have an 
opinion. If you have no opinion about a journal, put an X in the space. Use the 
Management Information Systems Quarterly as your standard of reference. A weight of 10 
has been arbitrarily assigned to M1SQ. A publication that you regard as half the worth of 
MISQ would get a 5. A publication you deem to be twice the value of MISQ would get a 20. 
And so on. Negative values are acceptable 

You may adopt any perspective or criteria you wish as the yardstick by which you are 
assessing these journals, e.g. value to the IS community, impact of published articles, quality 
of research, standing of authors, etc. The list of IS journals on the accompanying score sheet 
has been adopted from Davis(1980), Hamilton & Ives(1983) and Vogel & Wetherbe(1984). 


Thank You. 

William James Haga 

Associate Professor, Information Systems 




WEIGHT 


JOURNAL 

Academy of Management Journal 

Accounting Review 

ACM Computing Surveys 

ACM Transactions (various specialties) 

Administrative Science Quarterly 

Communications of the ACM 

Data Base 

Datamation 

Decision Sciences 

Decision Support Systems 

EDP Analyzer 

Harvard Business Review 

JFJRE Computer 

tfff Transactions (various specialties) 
Information and Management 
Information Systems Research 
Interfaces 

Journal of Management Information Systems 
Journal of Systems Management 
Management Information Systems Quarterly 
Management Science 
Omega 

ORSA Journal on Computing 
Sloan Management Review 
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APPENDIX D 


The following describes each academic specialty: 

Academic Specialty 8.1: Design and Evaluation of 
Information Systems, (The development of approaches for the 
analysis, design, specification, and evaluation of computer- 
assisted information systems). 

Academic Specialty 8.2: The Interaction of Information 
Systems, (The investigation of the relationships and 
interactions among four major components: information 
systems, information technology, organizations and society. 
The focus is on the interrelationships, not on the 
components themselves). 

Academic Specialty 8.3: Decision Support Systems, (The 
development of approaches for applying information systems 
technology to increase the effectiveness of decision-makers 
in situations where the computer system can support and 
enhance human judgements in the performance of tasks that 
have elements which cannot be specified in advance). 

Academic Specialty 8.4: Office Information Systems, (The 
study and development of information systems for office 
work. Such systems are concerned with the support of, and 
communication in connection with, human activities in an 
organization. They are characterized by, among other 
things, variety, informality and irregularity, but often 
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interact strongly with the more orderly, formal and 
predictable computer-based information systems used in that 
organization). 

Academic Specialty 8.5: Information Systems in Public 
Administration, (Information systems in public 
administration at international, national, regional, and 
local levels. The relationship between central and local 
use of information systems, and the provision of citizen 
services, together with the accomplishment of social goals). 
Again, academic specialties were not listed in all cases, 
hence the unknowns. 
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